SKaMPI: A Detailed, Accurate MPI Benchmark

SKaMPI is a benchmark for MPI implementations Its pur pose is the detailed analysis of the runtime of individual MPI operations and comparison of these for di erent implementations of MPI SKaMPI can be con gured and tuned in many ways operations measurement pre cision communication modes packet sizes number of processors used etc The technically most interesting feature of SKaMPI are measure ment mechanisms which combine accuracy e ciency and robustness Postprocessors support graphical presentation and comparisons of di er ent sets of results which are collected in a public web site We describe the SKaMPI design and implementation and illustrate its main aspects with actual measurements The Role of Benchmarking in Parallel Programming The primary purpose of parallel as opposed to sequential programming is achieving high processing speed There are two basic approaches to parallel pro gramming The high level approach tries to maintain ease of programming as much as possible and is exempli ed in languages such as HPF The low level approach attempts to achieve maximum speed the programmer devises all dis tribution of data and processing explicitly and manually optimizes the program for the idiosyncratic behavior of a particular target machine This approach is exempli ed by message passing programming Historically message passing programs were very machine speci c and hardly portable libraries such as PVM and MPI overcome this de ciency However one must not assume that functional portability also means performance portability Often what works on one machine will still work on another but be relatively much slower Worse yet the same may also happen when switching from one MPI implementation to another on the same machine Signi cant investments into performance evaluation tools have been made which can help to identify optimization possibilities However these tools have a number of limitations They require all target con gurations to be available for development which is often not the case if a program has to perform well on several platforms or if production runs involve more processors than available c by Springer Verlag Appeared in Recent Advances in Parallel Virtual and Message Passing Interface th European PVM MPI Users Group Meeting Liverpool UK September Lecture Notes in Computer Science for development Also these tools can only be applied in late development when the performance critical parts of the program are already working Finally the measurement process itself can distort the results in particular for the most challenging applications which use many processors and ne grained communi cation Therefore we consider a benchmark suite covering all important communica tion functions to be essential for developing e cient MPI programs It must be easy to use and produce detailed accurate and reliable results in a reasonable amount of time If standardized measurements for a wide spectrum of platforms are collected in a publicly available data base such a benchmark makes it pos sible to plan for portable e ciency already in the design stage thus leading to higher performance at lower development cost At the same time the benchmark must be exible enough to be adapted to special needs and expandable e g to incorporate the functionality of MPI The SKaMPI benchmark was designed with these goals in mind As a side e ect it can be used to compare di erent machines and MPI implementations Section provides an overview of the benchmark architecture The speci c capa bilities of SKaMPI and mechanisms for measurements are disussed in Section Section illustrates some of the techniques with actual measurement results